Tootfinder

Sleep-Like Unsupervised Replay Improves Performance when Data are Limited or Unbalanced
Anthony Bazhenov, Pahan Dewasurendra, Giri Krishnan, Jean Erik Delanois
https://arxiv.org/abs/2402.10956

Sleep-Like Unsupervised Replay Improves Performance when Data are Limited or Unbalanced
The performance of artificial neural networks (ANNs) degrades when training data are limited or imbalanced. In contrast, the human brain can learn quickly from just a few examples. Here, we investigated the role of sleep in improving the performance of ANNs trained with limited data on the MNIST and Fashion MNIST datasets. Sleep was implemented as an unsupervised phase with local Hebbian type learning rules. We found a significant boost in accuracy after the sleep phase for models trained with …

Brain-inspired Distributed Memorization Learning for Efficient Feature-free Unsupervised Domain Adaptation
Jianming Lv, Depin Liang, Zequan Liang, Yaobin Zhang, Sijun Xia
https://arxiv.org/abs/2402.14598 https://arxiv.org/pdf/2402.14598
arXiv:2402.14598v1 Announce Type: new
Abstract: Compared with gradient based artificial neural networks, biological neural networks usually show a more powerful generalization ability to quickly adapt to unknown environments without using any gradient back-propagation procedure. Inspired by the distributed memory mechanism of human brains, we propose a novel gradient-free Distributed Memorization Learning mechanism, namely DML, to support quick domain adaptation of transferred models. In particular, DML adopts randomly connected neurons to memorize the association of input signals, which are propagated as impulses, and makes the final decision by associating the distributed memories based on their confidence. More importantly, DML is able to perform reinforced memorization based on unlabeled data to quickly adapt to a new domain without heavy fine-tuning of deep features, which makes it very suitable for deploying on edge devices. Experiments based on four cross-domain real-world datasets show that DML can achieve superior performance of real-time domain adaptation compared with traditional gradient based MLP with more than 10% improvement of accuracy while reducing 87% of the timing cost of optimization.

Brain-inspired Distributed Memorization Learning for Efficient Feature-free Unsupervised Domain Adaptation
Jianming Lv, Depin Liang, Zequan Liang, Yaobin Zhang, Sijun Xia
https://arxiv.org/abs/2402.14598 https://arxiv.org/pdf/2402.14598
arXiv:2402.14598v1 Announce Type: new
Abstract: Compared with gradient based artificial neural networks, biological neural networks usually show a more powerful generalization ability to quickly adapt to unknown environments without using any gradient back-propagation procedure. Inspired by the distributed memory mechanism of human brains, we propose a novel gradient-free Distributed Memorization Learning mechanism, namely DML, to support quick domain adaptation of transferred models. In particular, DML adopts randomly connected neurons to memorize the association of input signals, which are propagated as impulses, and makes the final decision by associating the distributed memories based on their confidence. More importantly, DML is able to perform reinforced memorization based on unlabeled data to quickly adapt to a new domain without heavy fine-tuning of deep features, which makes it very suitable for deploying on edge devices. Experiments based on four cross-domain real-world datasets show that DML can achieve superior performance of real-time domain adaptation compared with traditional gradient based MLP with more than 10% improvement of accuracy while reducing 87% of the timing cost of optimization.